Search for: All records

New international academic collaborations are being created at a fast pace, generating data sets each day, in the order of terabytes in size. Often these data sets need to be moved in real-time to a central location to be processed and then shared. In the field of astronomy, building data processing facilities in remote locations is not always feasible, creating the need for a high bandwidth network infrastructure to transport these data sets very long distances. This network infrastructure normally relies on multiple networks operated by multiple organizations or projects. Creating an end-to-end path involving multiple network operators, technologies and interconnections often adds conditions that make the real-time movement of big data sets challenging. The Large Synoptic Survey Telescope (LSST) is an example of astronomical applications imposing new challenges on multi-domain network provisioning activities. The network for LSST is challenging for a number of reasons: (1) with the telescope in Chile and the archiving facility in the USA, the network has a high propagation delay, which affects traditional transport protocols performance; (2) the path is composed of multiple network operators, which means that the different network operating teams involved must coordinate technologies and protocols to support all parallel data transfers in an efficient way; (3) the large amount of data produced (12.7GB/image) and the small interval available to transfer this data (5 seconds) to the archiving facility requires special Quality of Service (QoS) policies; (4) because network events happen, the network needs to be prepared to be adjusted for rainy days, where some data types will be prioritized over others. To guarantee data transfers will happen within the required interval, each network operator in the path needs to apply QoS policies to each of its network links. These policies need to be coordinated end-to-end and, in the case where the network is affected by parallel events, all policies might need to be dynamically reconfigured in real-time to accommodate specific QoS policies for rainy days. Reconfiguring QoS policies is a very complex activity to current network protocols and technologies, sometimes requiring human intervention. This presentation aims to share the efforts to guarantee an efficient network configuration capable of handling LSST data transfers in sunny and rainy days across multiple network operators from South to North America.